Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Idomura, Yasuhiro; Ina, Takuya*; Ali, Y.*; Imamura, Toshiyuki*
Proceedings of International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2020) (Internet), p.1318 - 1330, 2020/11
Times Cited Count：1 Percentile：37.17(Computer Science, Information Systems)The multi-scale full-
simulation of the next generation experimental fusion reactor ITER based on a five dimensional (5D) gyrokinetic model is one of the most computationally demanding problems in fusion science. In this work, a Gyrokinetic Toroidal 5D Eulerian code (GT5D) is accelerated by a new mixed-precision communication-avoiding (CA) Krylov method. The bottleneck of global collective communication on accelerated computing platforms is resolved using a CA Krylov method. In addition, a new FP16 preconditioner, which is designed using the new support for FP16 SIMD operations on A64FX, reduces both the number of iterations (halo data communication) and the computational cost. The performance of the proposed method for ITER size simulations with 0.1 trillion grids on 1,440 CPUs/GPUs on Fugaku and Summit shows 2.8x and 1.9x speedups respectively from the conventional non-CA Krylov method, and excellent strong scaling is obtained up to 5,760 CPUs/GPUs.
Idomura, Yasuhiro
no journal, ,
To study fusion plasma turbulence, the Gyrokinetic Toroidal 5D full-f Eulerian code GT5D has been developed. On the K-computer, inter-node parallelization techniques such as multi-dimensional/-layer domain decomposition and communication-computation overlap were developed, and strong scaling of GT5D was improved up to 73,728 nodes. However, extensions of GT5D towards burning plasmas including kinetic electrons and multi-species ions require greater computing power. Under the post-K project, we have developed computing techniques for the next generation platforms such as GPUs and MICs. In this talk, we discuss computational challenges related to complicated intra-node memory hierarchy on many core processors and relatively limited inter-node communication performance compared with accelerated computation.
Idomura, Yasuhiro
no journal, ,
We discuss Exa-scale computing techniques for the gyrokinetic toroidal 5D full-f Eulerian code GT5D, which have been developed under the Post-K project. Towards burning plasma simulations in ITER, we have extended physics models including kinetic electrons and multi-species ions. To compute the burning plasma model at the ITER scale, we need Exa-scale computing. To this end, techniques for utilizing many core processors with low power consumption and avoiding communication bottlenecks, which are revealed by accelerated computation. In this talk, we explain many core optimization techniques, communication overlap techniques, and communication-avoiding algorithms, which have been developed to resolve the above issues, and show their performance evaluations on the latest many core platforms such as the Plasma Simulator.
Idomura, Yasuhiro
no journal, ,
We discuss Exa-scale computing techniques, which are developed under the Post-K project. Since fusion plasma simulations require first principles based computation of a convection-diffusion simulation in five dimensional phase space, Exa-scale computation is needed for analyzing the next generation experimental reactor ITER. To this end, techniques for utilizing many core processors with low power consumption and avoiding communication bottlenecks, which are revealed by accelerated computation. In this talk, we explain many core optimization techniques, communication-computation overlap techniques, and communication-avoiding algorithms, which have been developed to resolve the above issues, and show their performance evaluations on the latest many core platforms.
Idomura, Yasuhiro
no journal, ,
In this talk, we review the current status of the sub-project D "Core design of fusion reactor" in the Priority Issue on Post-K computer (Accelerated Development of Innovative Clean Energy Systems). In the project, an interdisciplinary research team consisting of experts on applied mathematics, computer science, plasma theory, and plasma experiment has addressed (1) development of exascale computing technologies, (2) extension of physics models for burning plasmas, and (3) V&V of developed codes against large scale experiments such as JT-60 and LHD. Outcomes from the first two years of the project are presented focusing on computing techniques and physics models for analyzing multi time scale phenomena such as transient responses of plasma turbulence against auxiliary heating and intermittent bursts of energetic particles driven modes, and future issues in the latter half of the project.
Idomura, Yasuhiro; Ina, Takuya*; Obrejan, K.; Asahi, Yuichi*; Matsuoka, Seikichi*; Imamura, Toshiyuki*
no journal, ,
Under the post-K project, we have developed computing techniques of the Gyrokinetic Toroidal 5D full-f Eulerian code GT5D towards the next generation computing platforms based on many core processors. We discuss computational challenges related to complicated intra-processor memory hierarchy and limited inter-node communication performance compared with accelerated computation. The former issue is addressed by optimizing data access patterns of a stencil kernel on each many core architecture, and high performance gains are obtained. The latter issue is resolved by using advanced communication avoiding Krylov methods, which enables an order of magnitude reduction of collective communications and improves arithmetic intensity of main computing kernels. By applying these novel computing techniques, the performance of GT5D is dramatically improved on the latest many core platforms, and excellent strong scaling up to the full system size of the Oakforest-PACS (8,192 KNLs) is achieved.
Idomura, Yasuhiro
no journal, ,
In this overview talk, first, we review computational nuclear engineering research activities at JAEA, which cover a wide spectrum of computing needs including various CFD codes on thermal hydraulics and environmental dynamics, radiation transport codes, quantum simulations on material science, and the latest machine learning applications. Second, we talk about the development of exascale nuclear CFD simulations on state-of-the-art accelerated computing platforms. Third, we present exascale plasma codes and new data science approaches developed under the post-K project. Finally, we discuss the state of the CEA-JAEA collaboration on computational nuclear engineering.
Idomura, Yasuhiro
no journal, ,
Under the Post-K project, a Gyrokinetic Toroidal 5D full-f Eulerian code GT5D has been developed towards exascale burning plasma simulations on the Post-K machine. In this talk, we review the present status on new computational techniques on GT5D. The main computing part of GT5D is given by Krylov based sparse matrix solvers for a semi-implicit time integration. We have ported computing kernels of these solvers on a prototype machine of FUGAKU and on V100 GPU, and confirmed that almost ideal performance gains are achieved on these state-of-the-art many core and GPU architectures.
Idomura, Yasuhiro; Ali, Y.*; Ina, Takuya*; Imamura, Toshiyuki*
no journal, ,
Implicit finite difference solvers based on Krylov subspace methods occupy dominant computing costs in the Gyrokinetic Toroidal 5D full-f Eulerian code GT5D. Under the post-K project, advanced communication avoiding (CA) Krylov subspace methods have been developed for exascale computing platforms, which have limited inter-node communication performance compared with accelerated computation. In this work, we develop a new mixed precision CA-GMRES solver using a FP16 preconditioner, which dramatically reduces the number of iterations, and thus, halo data communications. We port the new solver on FUGAKU and Summit, and compare its performance against conventional solvers on existing muti/many-core processors.
Idomura, Yasuhiro
no journal, ,
The Gyrokinetic Toroidal 5D full-f Eulerian code GT5D is based on a semi-implicit finite difference scheme, in which a stiff linear 4D convection operator is subject to implicit time integration, and the implicit finite difference solver for fast kinetic electrons occupies more than 80% of the total computing cost. The implicit solver was originally developed using a Krylov subspace method, in which global collective communications and halo data communications were becoming bottlenecks on the latest accelerator based platforms. To resolve this issue, the convergence property is improved by using a new FP16 preconditioner, and an order of magnitude reduction of the number of iterations and thus, communications was achieved. A communication-avoiding (CA) solver based on the FP16 preconditioner was developed by utilizing the new support for FP16 SIMD operations on FUGAKU, and was ported also on Summit. The new CA solver showed significant speedups both on FUGAKU and SUMMIT, and its performance portability was demonstrated.
Idomura, Yasuhiro
no journal, ,
The multi-phase multi-component thermal hydraulics code JUPITER, the urban wind analysis code CityLBM, and the in-situ visualization software In-Situ PBVR were ported and optimized on Fugaku, and their performances were evaluated. The performance of JUPITER, which is implemented in C, was compared against Oakforest-PACS, and a reasonable speedup due to the improved hardware was confirmed. In addition, the performance was further improved by using FP16 SIMD operations on A64FX. On the other hand, Fujitsu C compiler had many optimization issues on CityLBM, which is implemented in C++, and its performance was two orders of magnitudes slower than Tsubame, when the same number of computing nodes are used. Finally, we constructed an in-situ visualization environment on Fugaku using In-Situ PBVR, which was released as an open source software.
Watanabe, Tomohiko*; Idomura, Yasuhiro; Todo, Yasushi*; Honda, Mitsuru*
no journal, ,
During the initial start-up phase of the supercomputer Fugaku, we have launched a simulation project to explore physics of burning plasma confinement, that is, turbulent transport of particles, momentum, energy, impurity ions and hydrogen isotopes, and confinement of energetic particles, in collaboration with data science approaches. In prior to the project, we have upgraded and optimized three major fusion plasma simulation codes, GKV, GT5D and MEGA, which solve the kinetic plasma dynamics on multi-dimensional phase space, achieving the high computational performance on Fugaku. The flux tube gyrokinetic code, GKV, is applied to simulations of the multi-scale turbulence in multiple ion species plasma and the turbulent transport of heavy impurity ions. The global full-f gyrokinetic code, GT5D, is employed to explore non-local turbulent transport and intrinsic plasma rotation. The kinetic-MHD hybrid code, MEGA, is used for studying confinement of energetic ions, and has also been extended to introduce kinetic dynamics of bulk ions. In the project, data science approaches are also promoted to improve transport modeling and efficiency of the simulation research.
Idomura, Yasuhiro
no journal, ,
The Gyrokinetic Toroidal 5D full-f Eulerian code GT5D is accelerated on the world largest CPU based and GPU based supercomputers, Fugaku and Summit. GT5D is based on a semi-implicit finite difference scheme, in which the implicit finite difference solver for fast kinetic electrons occupies more than 80% of the total computing cost. The implicit solver was originally developed using a Krylov subspace method, in which global collective communications and halo data communications were becoming bottlenecks with accelerated computing. To resolve this issue, the convergence property is improved by using a new FP16 preconditioner, and an order of magnitude reduction of the number of iterations and thus, communications was achieved. The new solver showed significant speedups both on Fugaku and Summit, and enabled numerical experiments with real mass kinetic electrons.
